48 research outputs found
Bayesian Learning for Neural Networks: an algorithmic survey
The last decade witnessed a growing interest in Bayesian learning. Yet, the
technicality of the topic and the multitude of ingredients involved therein,
besides the complexity of turning theory into practical implementations, limit
the use of the Bayesian learning paradigm, preventing its widespread adoption
across different fields and applications. This self-contained survey engages
and introduces readers to the principles and algorithms of Bayesian Learning
for Neural Networks. It provides an introduction to the topic from an
accessible, practical-algorithmic perspective. Upon providing a general
introduction to Bayesian Neural Networks, we discuss and present both standard
and recent approaches for Bayesian inference, with an emphasis on solutions
relying on Variational Inference and the use of Natural gradients. We also
discuss the use of manifold optimization as a state-of-the-art approach to
Bayesian learning. We examine the characteristic properties of all the
discussed methods, and provide pseudo-codes for their implementation, paying
attention to practical aspects, such as the computation of the gradient
Tensor Representation in High-Frequency Financial Data for Price Change Prediction
Nowadays, with the availability of massive amount of trade data collected,
the dynamics of the financial markets pose both a challenge and an opportunity
for high frequency traders. In order to take advantage of the rapid, subtle
movement of assets in High Frequency Trading (HFT), an automatic algorithm to
analyze and detect patterns of price change based on transaction records must
be available. The multichannel, time-series representation of financial data
naturally suggests tensor-based learning algorithms. In this work, we
investigate the effectiveness of two multilinear methods for the mid-price
prediction problem against other existing methods. The experiments in a large
scale dataset which contains more than 4 millions limit orders show that by
utilizing tensor representation, multilinear models outperform vector-based
approaches and other competing ones.Comment: accepted in SSCI 2017, typos fixe
Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning
We develop an optimization algorithm suitable for Bayesian learning in
complex models. Our approach relies on natural gradient updates within a
general black-box framework for efficient training with limited model-specific
derivations. It applies within the class of exponential-family variational
posterior distributions, for which we extensively discuss the Gaussian case for
which the updates have a rather simple form. Our Quasi Black-box Variational
Inference (QBVI) framework is readily applicable to a wide class of Bayesian
inference problems and is of simple implementation as the updates of the
variational posterior do not involve gradients with respect to the model
parameters, nor the prescription of the Fisher information matrix. We develop
QBVI under different hypotheses for the posterior covariance matrix, discuss
details about its robust and feasible implementation, and provide a number of
real-world applications to demonstrate its effectiveness